MEMORANDUM December 18, 2015 


TO: Board Members 


FROM: Terry B. Grier, Ed.D. 
Superintendent of Schools 


SUBJECT: 2013-2014 ASPIRE Award Survey, Spring 2015 
CONTACT: Carla Stevens, 713-556-6700 


The purpose of the ASPIRE Award Survey was to gain insight regarding the level of knowledge 
and perceptions of Houston Independent School District (HISD) teachers and staff after nine 
years of implementation of growth-based performance pay in HISD, as well as their perceptions 
regarding the overall concept of teacher performance pay. Additionally, participants had the 
opportunity to provide recommendations to improve the ASPIRE Award program. 


e Of the 18,364 Houston Independent School District (HISD) campus-based employees 
surveyed, there were 4,031 participants who responded to the survey (22.0 percent) 
administered in December 2014. The response rate is fairly low and the results, while 
informative, may not be generalized to the population. 


e When comparing survey results over the last nine years, there was a decrease in the 
percent of respondents who were in favor or somewhat in favor of the concept of teacher 
performance pay from 69.2 percent in December 2007 to 49.7 percent in December 2014. 
When comparing January 2014 to December 2014, there has been a decrease in the 
respondents that indicated they were in favor or somewhat in favor of the concept of teacher 
performance pay by 3.5 percentage points. 


e Based on survey data collected in December 2014, the largest percentage of respondents 
(74.7 percent) indicated that over the past several years, they always collaborated with their 
colleagues. 


e When comparing the percentage of respondents that indicated they were in favor or 
somewhat in favor toward the concept of the Teacher-Performance Pay Model and to the 
ASPIRE Award Program for that year, it was first reported at 44.4 percent (December 2007 
survey administration), reached a peak of 53.3 percent in 2009, and was most recently 
reported at 39.5 percent (December 2014 survey administration), down from 46.5 percent in 
January 2014. 


e Based on December 2014 results, a higher percentage of respondents strongly disagreed or 
disagreed that their maximum award amount was commensurate with their professional 
contribution, 48.1 percent, compared to 21.4 percent who were neutral and 30.5 percent 
who agreed or strongly agreed. 


e Out of a total of 4,031 respondents on the December 2014 survey, 1,724 or 42.8 percent of 
the respondents provided at least one response for improving the 2013-2014 ASPIRE 
Award model. The six highest emergent categories based on the percentage of the 
responses centered on the following: make the model equitable, fair, transparent, and 


inclusive with clear expectations (12.8 percent); unintended consequences (divisive, 
cheating, free-riding) (11.9 percent); how the money should be allocated/reallocated (11.4 
percent); factors perceived as impacting growth or the calculation of growth (6.9 percent); 
same earning opportunity/award is not commensurate with professional duties (6.0 percent); 
discontinue the award (5.5 percent). 


Should you have any questions or need further information, please contact my office or Carla 


Stevens in Research and Accountability at 713-556-6700. 
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ASPIRE AWARD SURVEY RESULTS, Spring 2015 
FINDINGS RELATED TO TEACHER PERCEPTIONS, KNOWLEDGE, AND 
INPUT, 2013-2014 Award Model Payout 


Executive Summary 


Program Description 

On January 12, 2006, the Houston Independent School District (HISD) Board of Education approved the 
Teacher Performance-Pay Program (TPPM) awarding teachers financial incentives based on three strands 
of performance pay. These strands involved campus-level performance on the state accountability rating and 
individual teacher performance on the basis of student progress on state and district assessment programs. 
The awards were paid out in January, 2007. The experience gained in the first year and consultations with 
national experts and teachers provided the impetus for recommending the improvement and enhancement 
of the model which then became the award program for the district's school improvement framework, 
“Accelerating Student Progress: Increasing Results and Expectations” (ASPIRE). The ASPIRE Award 
program has completed its eighth year of payout, occurring in February 2015 (the ninth payout for 
performance pay in the district). 


This report provides the results of an annual survey administered on December 2014 designed to collect 
perceptions and input from HISD teachers and staff after nine years of implementation of growth-based 
performance pay (see the 2005-2006 Teacher Performance-Pay and 2006-2007 ASPIRE Award Survey; 
2007-2008 ASPIRE Award Survey, Spring 2009; 2008-2009 ASPIRE Award Survey, Spring 2010; 2009- 
2010 ASPIRE Award Survey, Spring 2011; 2010-2011 ASPIRE Award Survey, Spring 2012; 2011-2012 
ASPIRE Award Survey, Spring 2013; 2012-2013 ASPIRE Award Survey, Spring 2014 for previous results). 
This report addresses the district's strategic plan of providing an effective teacher in every classroom. Survey 
data focused on eight areas of interest that include: 


e Background characteristics of survey respondents; 


e Perceptions of respondents regarding the concept of teacher performance pay and the teacher 
appraisal system; 


e Perceptions of respondents regarding their level of agreement to specific instructional practices or 
behaviors encouraged by the ASPIRE Award program; 


e Perceptions of respondents and level of understanding of respondents regarding the Teacher 
Performance-Pay Model (TPPM) and the ASPIRE Award program; 


e Percent of respondents that watched the learning modules on value-added; 
e Effectiveness of communicating information about the ASPIRE Award; 


e Perceptions of respondents regarding their level of compensation and the ASPIRE Award model; 
and, 


e Recommendations for changing the 2013-2014 ASPIRE Award suggested by respondents. 
Highlights 


e Of the 18,364 Houston Independent School District (HISD) campus-based employees invited to 
participate, 4,031 participants (22.0 percent) responded to the survey administered in December 
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2014. Of the 2,972 respondents who indicated an award category, 58.1 percent were core teachers 
(Groups 1-3), 12.0 percent were elective/ancillary teachers, 8.7 percent were instructional support 
staff, 7.9 percent were teaching assistants, 8.4 percent were operational support staff, and 4.9 
percent were either principals or assistant principals/deans of instruction. Any conclusions drawn 
from this survey should be made with caution given the low response rate. 


e When comparing survey results over the last nine years, there was a decrease in the percent of 
respondents who were in favor or somewhat in favor of the concept of teacher performance pay from 
69.2 percent in December 2007 to 49.7 percent in December 2014. When comparing January 2014 
to December 2014, there has been a decrease in the respondents that indicated they were in favor 
or somewhat in favor of the concept of teacher performance pay by 3.5 percentage points. 


e Based on survey data collected in December 2014, the largest percentage of respondents (74.7 
percent) indicated that over the past several years, they always collaborated with their colleagues. 


e When comparing the percentage of respondents that indicated they were in favor or somewhat in 
favor toward the 2005-2006 Teacher-Performance Pay Model or to the specific ASPIRE Award 
program for that year, it was first reported at 44.4 percent (December 2007 survey administration), 
reached a peak of 53.3 percent in 2009, and was most recently reported at 39.5 percent (December 
2014 survey administration), down from 46.5 percent in January 2014. 


e When comparing survey results from January 2014 to December 2014, there was a decrease to 28.8 
percent in the percentage of respondents that indicated their level of understanding of the ASPIRE 
Award Program was very high or high (7.8 percentage points), as well as an increase to 26.1 percent 
in the percentage of respondents that indicated their level of understanding of the ASPIRE Award 
Program was low/very low (6.8 percentage points). 


e Based on the December 2014 survey results, the respondents who indicated they were Group 1 Core 
Teacher Grades 3-10 with EVAAS ® and had watched the EVAAS® Learning Modules in the past 
twelve months indicated that they had a high/very high level of understanding of value-added analysis 
(35.5 percent) compared to those Group 1 Core Teacher Grades 3—10 with EVAAS ® that had not 
watched any Learning Modules (23.2 percent) or who indicated they were unaware of the 
aforementioned resource (11.1 percent). 


e Based on survey results from March 2010 and December 2014, 26.7 percent and 32.2 percent of 
respondents, respectively strongly agreed or agreed that the ASPIRE Award is a fair way of 
acknowledging a teacher's impact on student growth. 


e Based on December 2014 results, a higher percentage of respondents strongly disagreed or 
disagreed that their maximum award amount was commensurate with their professional contribution, 
48.1 percent, compared to 21.4 percent who were neutral and 30.5 percent who agreed or strongly 
agreed. 


e Although a majority of respondents indicated that communication was moderately effective or very 
effective for providing clear explanations about the award model (51.0 percent), providing clear 
explanations about value-added calculations (45.5 percent) and providing clear explanations about 
comparative growth calculations (46.4 percent), these items were noticeably lower than for other 
areas surveyed (57.6 percent to 69.2 percent). 


e Out of a total of 4,031 respondents on the December 2014 survey, 1,724 or 42.8 percent of the 
respondents provided at least one response for improving the 2013-2014 ASPIRE Award model. 
The six highest emergent categories based on the percentage of the responses centered on the 
following: make the model equitable, fair, transparent, and inclusive with clear expectations (12.8 
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percent); unintended consequences (divisive, cheating, free-riding) (11.9 percent); how the money 
should be allocated/reallocated (11.4 percent); factors perceived as impacting growth or the 
calculation of growth (6.9 percent); same earning opportunity/award is not commensurate with 
professional duties (6.0 percent); discontinue the award (5.5 percent). 


Administrative Response 
The district continues to use the information from the ASPIRE Award program evaluation and the 
ASPIRE Award survey to make annual improvements to the ASPIRE Award model. 
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Introduction 


The purpose of the ASPIRE Award Model is to reward teachers for their efforts in improving the academic 

growth of their students. ASPIRE Award employs a value-added methodology that provides teachers with the 

information that they need to facilitate and measure student progress at the student, classroom, and campus 

levels. The ASPIRE Award is dedicated to achieving the following goals: 

e Encourage cooperation in Professional Learning Communities; 

e Be aligned with the district's other school-improvement initiatives; 

e Use value-added data based on a national expert's methodology to reward teachers reliably and 
consistently for student progress; and 

e Include core teachers at all grade levels, early childhood through grade 12. 


The ASPIRE Award is based on the following principles: 

e Performance pay drives academic performance; 

e Good teaching occurs in all schools; 

e Teamwork is valuable; 

e Performance pay does not replace a competitive base salary, and 
e Performance pay systems are dynamic and evolve over time. 


Given these goals and principles, the ASPIRE Award involves three different indicators of academic 
performance: Indicator |-Individual Performance: (value-added core teacher progress); Indicator II-Group 
Performance: Teachers (department value-added or comparative growth); and Indicator III-Group 
Performance: Campus-Wide (campus value-added and campus growth or achievement). Indicator III is based 
on the EVAAS campus composite cumulative gain index and the Stanford and Aprenda reading and 
mathematics performance (percent of all students at/above 50" national percentile rank, across all grades) 
for middle and elementary schools, and Advanced Placement (AP)/International Baccalaureate (IB) 
participation and performance for high schools. Under the model, every HISD teacher has the opportunity to 
participate in at least Indicator III. 


Methods 


Data Collection and Analysis 


e The ASPIRE award survey items were developed from previous surveys, reviewed, and approved by 
members of the ASPIRE Award Executive Committee with input from the Department of Human 
Resources and Professional Educator Compensation and Support (PECAS) Committee. The 2013-— 
2014 ASPIRE Award Survey was administered on-line from Wednesday, December 3, 2014 to 
Friday, December 19, 2014, with follow-up reminders on Thursday, December 11, 2014 and 
Thursday, December 18, 2014. The survey responses were completely anonymous through 
SurveyMonkey with no IP addresses collected. The survey instructions with the embedded link to 
access the survey were sent directly to campus-based employees by HISD partner Battelle for Kids. 


e The data obtained from the completed surveys were downloaded from SurveyMonkey and analyzed 
using SPSS and Microsoft Access. Items that were skipped or for which respondents answered "N/A" 
were coded as missing data and not included in the analysis. The text analysis tool was used to 
identify emergent categories for the open-ended questions. To improve the response rate, 
respondents who completed the survey had the opportunity to be drawn for one of 9 $50.00 gift cards 
and one $150.00 gift card donated by the HISD Foundation. 
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Data Limitations 


Changes in the structure of the survey and coding practices limited comparisons to the results of 
previously developed survey instruments. For the December 2014 survey administration, data quality 
checks were conducted and corrections made regarding skip patterns and building level. Any 
conclusions from these results should be made with caution due to the low response rate. The 
responses may not be generalizable to the population of campus-based staff who were initially invited 
to participate. 


Results 


What were the background characteristics of survey respondents? 


Of the 18,364 Houston Independent School District (HISD) campus-based employees invited to 
complete the survey, there were 4,031 participants who responded to the survey (22.0 percent) 
administered in December 2014. Any conclusions drawn from this survey should be made with 
caution given the low response rate (Table 1, p. 26). 


Of the 4,031 respondents, 2,972 indicated their ASPIRE Award categorization for the 2013-2014 
school year. Core teachers (Group 1, 2, and 3) represented the highest percentage of respondents 
with 58.1 percent, followed by elective/ancillary teachers with 12.0 percent (Table 2, p. 26). 


The majority of respondents reported holding either a Bachelor's Degree (36.4 percent) or a Master's 
Degree (32.6 percent). The average experience in HISD was 10.3 years with the average experience 
at the current campus being 6.8 years (Table 3, p. 26). 


Approximately 83 percent of the respondents were employed in HISD for the 2013—2014 school year, 
and approximately 80.7 percent were eligible to receive an award. Fifty-nine percent of the 
respondents indicated that they will receive an ASPIRE Award, and 66.1 percent of core foundation 
teachers who responded received an individual performance award, an award based on teacher 
progress for the 2013-2014 school year (Table 4, p. 27). 


Of the 1,513 December 2007 survey respondents, 65.6 percent indicated that they received an 
award. The percentage continued to increase through the March 2011 survey, where 90.3 percent of 
respondents received an award. There was a decline of 10.2 percentage points from March 2011 to 
March 2012, with a 25.3 percentage point decline from March 2012 to January 2014, followed by an 
increase of 4.1 percentage points in December 2014 (Figure 1, p. 6). 
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Figure 1. Percent of respondents receiving an award based on results of nine survey 
administrations 
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e Over the past nine years, the percentage of survey respondents who reported receiving an award 
increased from 65.6 percent in 2007 to 90.3 in 2011, but declined by 31.4 percentage points from 
March 2011 to December 2014 (Figure 1). This may be due in large part to changes in the award 
model criteria. 


e Onthe December 2014 survey, respondents were asked to indicate if they taught in a critical shortage 
area. Since respondents may have taught in more than one critical shortage area, percentages are 
based on the total number of responses. Of the 3,244 responses, 11.8 percent indicated Special 
Education, 10.4 percent indicated Bilingual Education, 10.4 percent indicated English as a Second 
Language (ESL), 7.0 percent indicated secondary mathematics, 5.4 percent indicated secondary 
science, 1.2 percent indicated secondary Spanish, and 53.8 percent did not teach in a critical 
shortage area (N/A)(Table 5, p. 27). 


What were the perceptions of respondents regarding the concept of teacher performance pay overall 
and the teacher appraisal system? 


e When comparing survey results over the last nine years, there was an overall decrease in the percent 
of respondents who were in favor or somewhat in favor of the concept of teacher performance pay 
from 69.2 percent in December 2007 to 49.7 percent in December 2014 (Figure 2). 


e When comparing survey results over the last nine years, there was an overall increase in the percent 
of respondents who were somewhat opposed or opposed to the concept of teacher performance pay 
from 18.8 percent in December 2007 to 32.3 percent in December 2014 (Figure 2). 


e The percentage of campus-based staff in favor or somewhat in favor of the concept of teacher 
performance-pay increased to 58.6 percent after the 2012 payout, decreased to 51.7 percent after 
the 2013 payout, increased to 53.2 percent with the January 2014 payout, and decreased to 49.7 
percent prior to the February 2015 payout (Figure 2). 
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Figure 2. Percent of respondents indicating favorability toward the concept of performance pay over 
nine years 
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e When respondents on the December 2007 survey administration were asked how favorable they 
were toward the concept of teacher performance pay based on individual student growth, 62.2 
percent indicated they were in favor or somewhat in favor, compared to 40.0 percent of respondents 
in December 2014, reflecting the lowest level of agreement in nine years (Figure 3). 


e The percentage of survey respondents indicating that they were somewhat opposed or opposed 
toward the concept of teacher performance pay based on individual student growth increased over 
the 9-year period from 24.5 percent in 2007 to 42.1 percent in 2014 (Figure 3). 


Figure 3. Percent of respondents indicating favorability toward the concept of teacher performance 
pay based on individual student growth over nine years 
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e Over the past nine years, survey respondents were asked to indicate their perceptions about the 
concept of performance pay based on passing rates. When comparing overall survey results from 
December 2007 to December 2014, there was an increase in the percent of respondents indicating 
that they were somewhat opposed or opposed to teacher performance pay based on passing rates 
by 3.6 percentage points, and the largest percentage of respondents still remains opposed to using 
passing rates for performance pay (Figure 4). 


Figure 4. Percent of respondents indicating favorability toward the concept of teacher performance 
pay based on passing rates over nine years 
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e Over the past seven years, survey respondents were asked to indicate their perceptions about the 
concept of receiving differentiated pay as seen in Figure 5. The percentage of campus-based staff 
in favor or somewhat in favor of the concept of differentiated pay decreased overall from 55.5 percent 
after the 2009 payout to 48.1 percent in December 2014, prior to the 2015 payout. 


e Over the past seven years, the percent of respondents indicating that they were opposed or 
somewhat opposed to differentiated pay increased from 22.1 percent in 2009 to 28.9 percent in 
December 2014 (Figure 5). 
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Figure 5. Percent of respondents indicating favorability toward the concept of differentiated 
pay for the past seven years 
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e When comparing survey results from January 2014 to December 2014, data were collected on the 
favorability of respondents towards the concept of an award for educators in hard-to-staff buildings. 
The majority of respondents (57.0 percent and 67.0 percent ) indicated that they were in favor or 
somewhat in favor of awarding an incentive to educators in hard-to-staff buildings, with 25.2 percent 
and 20.7 percent who were neutral, and 17.9 percent and 12.3 percent who were somewhat opposed 
or opposed. Over the past two years, this item has inceased in favorability by 10.0 percentage points, 
from 57.0 percent in January 2014 to 67.0 percent in Decemer 2014 (Figure 6). 


Figure 6. Percent of respondents indicating favorability toward the concept of an award for 
educators in hard-to-staff building 
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e To determine whether there were differences in perceptions toward the concept of performance pay 
overall, comparisons were made between core foundation teachers and non-core instructional staff 
(December 2007 and December 2014) as summarized in Figure 7 (p. 10) Based on results of the 
December 2007 survey administration, the percentage of core foundation teachers who were in favor 


HISD Research and Accountability 9 


or somewhat in favor of teacher performance pay exceeded that of non-core instructional staff by 8.4 
percentage points; whereas, December 2014 survey results indicated that the percentage of core 
foundation teachers who were in favor or somewhat in favor of teacher performance pay was less 
than that of non-core instructional staff by 2.4 percentage points. Favorable responses have 
decreased overall for both groups over the last nine years. 


Figure 7. Percent of respondents indicating favorability toward the concept of teacher 
performance pay overall by core foundation and non-core instructional staff, December 2007 
and December 2014 
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Note: To make 2014 comparable to the 2007 survey administration data, non-instructional employees (ASPIRE Award 
Group 6 and 7) (N=485) and principals (ASPIREAward Group 1L) (N=74) were not included in this analysis. 


e Appendix A (p. 32) compares differences in perceptions toward the concept of teacher performance 
pay overall by eligibility category (December 2014). Of the respondents that indicated that they were 
eligible to receive an award and who indicated a particular eligibility category, 69.6 percent of 
principals indicated they were somewhat in favor or in favor toward the concept of teacher 
performance pay, reflecting the highest level of agreement of all the eligibility categories. This was 
followed by assistant principals/deans at 65.2 percent, operational support staff at 59.0 percent, and 
teaching assistants at 54.4 percent. 


e Of the respondents that indicated that they were eligible to receive an award and who indicated a 
particular eligibility category, 45.1 percent of elective/ancillary teachers indicated that they were 
somewhat opposed or opposed toward the concept of teacher performance pay, reflecting the highest 
level of disagreement to the statement. 


e For those respondents that reported they were Not Eligible to receive an ASPIRE award, 40.7 
percent were somewhat in favor or in favor and 41.1 percent were somewhat opposed or opposed 
toward the concept of teacher performance pay, not notably different from eligible teachers. 


e To determine whether there were differences in perceptions toward the concept of teacher 
performance pay based on individual student growth, comparisons were made between core 
foundation teachers and non-core instructional staff through time (December 2007 and December 
2014). Figure 8 summarizes the results. The percentage of core foundation teachers who were in 
favor or somewhat in favor of teacher performance pay based on individual student growth 
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exceeded that of non-core instructional staff by 11.6 percentage points based on December 2007 
results and only 1.6 percentage points based on December 2014 results. 


e The percentage of non-core instructional staff that indicated they were somewhat opposed or 
opposed toward the concept of teacher performance pay based on individual student growth 
exceeded that of core foundation teachers by 9.8 percentage points in December 2007 compared to 
the percentatge of core teachers that exceeded non-core instructional staff by 1.2 percentage points 
based on December 2014 results. 


Figure 8. Percent of respondents indicating favorability toward the concept of teacher performance 
pay based on individual student growth by core foundation and non-core instructional staff, 
December 2007 and December 2014 
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Note: To make 2014 comparable to the 2007 survey administration data, non-instructional employees (ASPIRE Award 
Groups 6 and 7) (N=485) and principals (ASPIREAward Group 1L) (N=74) were not included in this analysis. 


e To determine whether there were differences in perceptions between core foundation teachers and 
non-core instructional staff over time regarding favorability toward the concept of teacher 
performance pay based on passing rates, comparisons were made using results from the December 
2007 survey administration and the December 2014 survey administration. Figure 9 (p. 12) 
summarizes the results. The percent of core foundation teachers who were in favor or somewhat in 
favor of teacher performance pay based on passing rates only exceeded that of non-core 
instructional staff by 5.3 percentage points in December 2007 and by just 3.1 percentage points in 
December 2014. 


e Approximately 52 percent of core foundation teachers and non-core instructional staff indicated that 
they were somewhat opposed or opposed toward the concept of teacher performance pay based on 
passing rates for the December 2007 survey administration which increased to 58.4 percent of core 
foundation teachers and 59.2 percent of non-core instructional staff based on survey results from the 
December 2014 administration (Figure 9). 
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Figure 9. Percent of respondents indicating favorability toward the concept of teacher 
performance pay based on passing rates by core foundation and non-core instructional 
staff, December 2007 and December 2014 


100 


Percent 
ol 
oO 


2007 Core 2007 Non-Core 2014 Core 2014 Non-Core 
Core Foundation Teachers and Non-Core Instructional Staff 


Gln favor/Somewhat in Favor Neutral @ Opposed/Somewhat Opposed 


Note: To make 2014 comparable to the 2007 survey administration data, non-instructional employees (ASPIRE Award 
Groups 6 and 7) (N=485) and Principals (ASPIREAward Group 1L) (N=74) were not included in this analysis. 


e Appendix B (p. 33) summarizes the results by eligibility category regarding perceptions towards the 
concept of teacher performance pay based on individual student growth, Appendix C (p. 34) 
summarizes the results by eligibility category regarding perceptions towards the concept of teacher 
performance pay based on passing rates only, and Appendix D (p. 35) summarizes the results by 
eligibility category regarding perceptions towards the concept of differentiated pay based on the 
December 2014 survey administration. 


e Of the respondents that indicated that they were eligible to receive an award and who indicated a 
particular eligibility category, 75.4 percent of principals and 59.1 percent of assistant principals/deans 
of instruction indicated they were somewhat in favor or in favor toward the concept of teacher 
performance pay based on individual student growth, reflecting the highest levels of agreement of all 
the eligibility categories (Appendix B, p. 33). 


e Of the respondents that indicated that they were eligible to receive an award and who indicated a 
particular eligibility category, 59.9 percent of elective/ancillary teachers and 50.2 percent of core 
teachers, grades 3-12 w/o EVAAS®, indicated that they were opposed or somewhat opposed toward 
the concept of teacher performance pay based on individual student growth (Appendix B, p. 33). 


e Of the respondents that indicated that they were eligible to receive an award and who indicated a 
particular eligibility category, 39.1 percent of principals and 21.6 percent of elective/ancillary teachers 
indicated they were somewhat in favor or in favor toward the concept of teacher performance pay 
based on individual passing rates, reflecting the highest and lowest levels of agreement, respectively, 
of all the eligibility categories based on December 2014 results (Appendix C, p. 34). 
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Of the respondents that indicated that they were eligible to receive an award and who indicated a 
particular eligibility category, 63.5 percent of elective/ancillary teachers and 25.6 percent of teaching 
assistants indicated that they were opposed or somewhat opposed toward the concept of teacher 
performance pay based on passing rates, reflecting the highest and lowest levels of disagreement, 
respectively, of all of the eligibility categories (Appendix C, p. 34). 


Of the respondents that indicated that they were eligible to receive an award and who indicated a 
particular eligibility category, 68.7 percent of principals indicated they were somewhat in favor or in 
favor toward the concept of differentiated pay, reflecting the highest level of agreement of all the 
eligibility categories. This was followed by assistant principals/deans of instruction at 63.1 percent. 
Elective/ancillary and Group 2: Core Teacher PK—2 teachers had the lowest levels of agreement with 
only 40.6 percent in favor or somewhat in favor (Appendix D, p. 35). 


Of the respondents that indicated that they were eligible to receive an award and who indicated a 
particular eligibility category, 43.3 percent of elective/ancillary teachers indicated that they were 
somewhat opposed or opposed toward the concept of differentiated pay, reflecting the highest level 
of disagreement to the statement (Appendix D, p. 35). 


On the teacher appraisal system, participants were asked to rate the rigor of the system on a scale 
of 1 (not rigorous) to 5 (very rigorous) and fair on a scale of 1 (not fair) to 5 (very fair). A majority 
(56.0 percent) of respondents rated the system as rigorous while 22.1 percent rated the system as 
fair, with a rating of 4 (Agree) or 5 (Strongly Agree). 


What were the perceptions of respondents regarding their level of agreement to specific 
instructional practices or behaviors encouraged by the ASPIRE Award program or practiced over the 
past several years? 


Over the past six years, respondents were asked whether the ASPIRE Award encouraged specific 
behaviors. Table 6 (p. 27) compares the responses of respondents for four items to the baseline 
year. The largest percentage of respondents in 2009 indicated that they agreed or strongly agreed 
that the ASPIRE Award encouraged them to continue teaching in the classroom (47.9 percent), 
remain working in HISD (44.0 percent) (baseline year is 2012), and that the ASPIRE Award 
encouraged them to come to work on a daily basis (47.0 percent). These percentages decreased to 
37.3 percent, 39.2 percent, and 38.3 percent, respectively in December 2014 survey data. 


Based on survey data collected in December 2014, the largest percentage of respondents indicated 
that over the past several years, they collaborated with my colleagues (74.7 percent) always (Table 
7, p. 28). 


Based on survey data collected in December 2014, 6.6 percent of respondents indicated that they 
never used value-added data to make instructional decisions, compared to 41.5 percent who always 
used value-added data to make instructional decisions (Table 7, p. 28). 


What were the perceptions and level of understanding of respondents regarding the Teacher 
Performance-Pay Model (TPPM) and ASPIRE Award program? 


Figure 10 (p. 14) summarizes the perceptions of respondents towards the respective performance- 
pay models through time. When comparing the percentage of respondents that indicated they were 
in favor or somewhat in favor toward the 2005-2006 Teacher-Performance Pay Model and to the 
specific ASPIRE Award program for that year, it was first reported at 44.4 percent (December 2007 
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survey administration), reached a peak of 53.3 percent in 2009, and was most recently reported at 
46.5 percent (December 2014 survey administration). This is an 4.4 percent point increase from 
March 2013. These results were after the payout, or in the most recent survey adminsitration, prior 
to the payout of each model. 


e When comparing survey results after or just prior to each payout, the percentage of respondents that 
indicated they were somewhat opposed or opposed toward the 2005-2006 Teacher Performance- 
Pay Model and to the 2013-2014 ASPIRE Award program decreased by 11.1 percentage points over 
a nine-year period, with the low being in 2009 at 24.0 percent (Figure 10). 


Figure 10. Percent of survey respondents’ favorability toward the performance-pay model paid out 
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e Figure 11 (p. 15) summarizes the results regarding the level of understanding respondents indicated 
toward the ASPIRE award models for each of the last eight years. 


e When comparing survey results from May 2008 to December 2014, the percentage of respondents 
that indicated their level of understanding of the ASPIRE Award program was very low or low, 
increased by 8.7 percentage points, and there was a decrease in the percentage of respondents that 
indicated their level of understanding of the ASPIRE Award program was sufficient (10.1 percentage 
points) (Figure 11, p. 15). 


e When comparing survey results from January 2014 to December 2014, there was a decrease in the 
percentage of respondents that indicated their level of understanding of the ASPIRE Award program 
was very high or high (7.8 percentage points), as well as an increase in the percentage of 
respondents that indicated their level of understanding of the ASPIRE Award program was low/very 
low (6.8 percentage points) (Figure 11, p. 15). 
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Figure 11. Percent of survey respondents’ level of understanding of the performance-pay model 
paid out that year 
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To determine whether there were differences in perceptions regarding the level of understanding 
toward ASPIRE, comparisons by eligibility category for ASPIRE December 2014 respondents are 
summarized in Appendix E (p. 36). Based on respondent data from the nine eligibility categories, 
principals and assistant principals indicated having a very high/high level of understanding (60.6 
percent and 56.9 percent, respectively) compared to core teachers, elective/ancillary teachers, 
instructional support staff, teaching assistants, operational support staff, and those indicating that 
they were Not Eligible to receive an ASPIRE award (ranging from 26.2 percent for Operational 
Support Staff to 36.8 percent for Instructional Support Staff. 


On the December 2014 survey, 26.8 percent of respondents that indicated they were Group 3: Core 
Teachers Grades 3-12 without EVAAS® as well as respondents that indicated they were Not Eligible 
to receive an award perceived their level of understanding of the ASPIRE Award program as very 
low or low, reflecting the greatest lack of understanding for ASPIRE survey respondents (Appendix 
E, p. 36). 


What percentage of respondents watched Value-Added/EVAAS learning modules, and what was the 
level of understanding? 


Figure 12 provides a comparison of the percent of respondents who watched at least one Value- 
Added/EVAAS Learning Module in the past 12 months. Out of 3,197 respondents, 31.1 indicated 
Yes, 40.9 percent responded No, and 28.0 percent indicated that they did not know this resource was 
available. 
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Figure 12. Percent of survey respondents watching value-added learning modules 
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e When comparing the perceptions of respondents from May 2008 to December 2014, there was a 5.8 
percentage point decrease regarding respondents that rated their level of understanding of the 
difference between student achievement and academic progress as very high or high (Figure 13). 


e Over the past eight years, the percent of respondents who rated their level of understanding of the 
difference between student achievement and academic progress as very low or low increased overall 
by 6.2 percentage points, although there was an overall decrease of 1.6 percentage points from 2008 
to 2012 (Figure 13). 


Figure 13. Percent of respondents indicating their level of understanding of the difference 
between student achievement and academic progress over eight years 
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On the May 2008 ASPIRE Award survey, there were seven items that were designed to determine 
the level of understanding for different training components related to the ASPIRE Award. Table 8 
(p. 28) depicts the comparison of the baseline data collected in May 2008 with data collected in 
December 2014. 


The percentage of respondents indicating a high/very high level of understanding decreased for six 
of the seven components. However, December 2014 had a lower number of respondents compared 
to 2008 (Table 8, p. 28). 


Based on survey data collected in May 2008 and December 2014, the training component for which 
the largest percentage of respondents indicated, in both years, a very high or high level of 
understanding centered on my understanding of the difference between student achievement and 
academic progress (44.5 and 38.7 percent, respectively) (Table 8, p. 28). 


Based on survey data collected in May 2008 and December 2014, the training component for which 
the largest percentage of respondents indicated, in both years, a very low or low level of 
understanding focused on how the ASPIRE Awards were calculated/determined (33.9 percent and 
44.1 percent, respectively) (Table 8, p. 28). 


One question asked respondents whether they perceived a connection between classroom 
instruction and performance-pay results. Figure 14 compares the percent of respondents from the 
past seven years’ surveys. Based on the May 2009 and December 2014 survey results, there was a 
decrease in the percentage of survey respondents who strongly agreed or agreed that there was a 
connection between classroom instruction and ASPIRE Award results (44.7 percent and 36.6 
percent, respectively). 


For the 2009 survey, 29.0 percent of the respondents disagreed or strongly disagreed with the 
statement that there was a connection between classroom instruction and ASPIRE Award results; 
however, this increased to 40.7 percent on the December 2014 survey (Figure 14). Nevertheless, 
these results showed slight improvements from the March 2013 responses. 


re 14, Percent of respondents indicating a connection between classroom instruction and 
ASPIRE Award results over seven years 
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e Across tabulation was conducted to determine whether there were differences in the level of 
understanding of value-added analysis who reported they were Group 1: Core Teacher Grades 3- 
10 with EVAAS® who reported watching value-added analysis learning modules and those that 
reported not watching or not aware of the resource. Figure 15 summarizes the results. 


e For the December 2014 survey administration, a higher percentage of respondents who watched the 
value-added analysis learning modules training reported a high or very high understanding of value- 
added analysis compared to those who did not watch the value-added learning moduels or who were 
unaware of the resource (35.5 percent, 23.2 percent, and 11.1, respectively) (Figure 15). 


e For December 2014 administration, 74.6 percent of respondents who watched the value-added 
analysis learning modules reported a sufficient or higher understanding of value-added analysis 
compared to those who did not watch the value-added training at 62.7 or who were unaware of the 
resource at 47.2 percent (Figure 15). 


Figure 15. Cross tabulation summarizing the percent of respondents indicating their level of 
understanding of value-added analysis and attending value-added training 
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Note: Items that were skipped were coded as missing data and not included in the analysis. 


What were the perceptions of respondents regarding their level of compensation and the ASPIRE 
Award Model? 


e There were seven items that were designed to examine the perceptions of respondents regarding 
the amount of money awarded and the ASPIRE model. The results from 2010 (three years ago) and 
2014 (most recent) are summarized in Table 9 (p. 29). 


e On the 2010 and 2014 survey administrations, the statement for which the largest percentage of 
respondents indicated strongly agree or agree centered on continuing the ASPIRE Award and 
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modifying the model on an annual basis (48.7 percent and 54.2 percent, respectively) (Table 9, p. 
29). 


e For the December 2014 administration, a higher percentage of respondents strongly disagreed or 
disagreed that their maximum award amount was commensurate with their professional contribution 
(48.1 percent) compared to 21.4 percent who were neutral and 30.5 percent who agreed or strongly 
agreed (Table 9, p. 29). 


e Perceptions from 2010 to 2014 became more positive and less negative on six of the seven items 
(Table 9, p. 29). 


e To determine whether there were differences in perceptions about the connection between classroom 
instruction and performance pay results, comparisons were made by eligibility category and 
respondents who indicated they were not eligible as summarized in Appendix F (p. 37). 


e For December 2014, the percentage of teaching assistants who strongly agreed or agreed that there 
was a connection between classroom instruction and the ASPIRE Award results (64.6 percent) 
exceeded core teachers (Groups 1-3), elective/ancillary teachers, instructional support staff, 
operational support staff, principals, assistant principals/deans, and those respondents that indicated 
they were not eligible to receive an award (Appendix F, p. 37). 


e The highest percentage of respondents that disagreed or strongly disagreed that there was a 
connection between classroom instruction and the ASPIRE Award results was from core teachers, 
grades 3-12 w/o EVAAS® (56.9 percent) (Appendix F, p. 37). 


e To determine whether there were differences in perceptions regarding the maximum award amount 
reflecting adequate recognition for efforts to increase student progress, comparisons were made by 
eligibility category and respondents who indicated they were not eligible as summarized in Appendix 
G (p.38). 


e For December 2014, 39.9 percent of core teachers grades 3-10 and 59.8 percent of teaching 
assistants, agreed or strongly agreed that their maximum ASPIRE Award adequately recognized their 
efforts to increase student progress, reflecting the highest levels of agreement compared to the 
remaining eligibility categories and for those respondents indicating they were not eligible to receive 
an award (Appendix G, p. 38). 


e For December 2014, 63.4 percent of elective/ancillary teachers, 62.4 percent of instructional support 
staff, and at least 54.0 percent of core teachers, grades 3-12 without EVAAS, core teachers PK-2, 
or respondents that indicated they were not eligible to receive an award indicated that they strongly 
disagreed or disagreed that their maximum ASPIRE Award adequately recognized their efforts to 
increase student progress (Appendix G, p. 38). 


e To determine whether differences existed with regard to the statement, the maximum award amount 
for my ASPIRE Award category is commensurate with my professional contribution, comparisons 
were made by eligibility category and for those respondents that indicated they were not eligible to 
receive an award. Appendix H (p. 39) summarizes the results. 


e For December 2014, 55.6 percent of teaching assistants agreed or strongly agreed that their 
maximum ASPIRE Award was commensurate with their professional contribution, reflecting the 
highest levels of agreement compared to the remaining eligibility categories and those respondents 
indicating they were not eligible to receive an award (Appendix H, p. 39). 


e On the December 2014 survey administration, 66.1 percent of elective/ancillary teachers, 64.5 
percent of assistant principals and deans, and 63.0 percent of instructional support staff indicated 
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that they strongly disagreed or disagreed that their maximum ASPIRE Award was commensurate 
with their professional contribution (Appendix H, p. 39). 


To determine whether there were differences in perceptions indicating favorability toward the concept 
of an award for educators in hard-to-staff buildings, comparisons were made by eligibility category 
and respondents who indicated they were not eligible as summarized in Appendix I (p.40). 


On the December 2014 survey administration, the majority of all eligibility categories as well as those 
that indicated they were not eligible to receive an award indicated that they strongly agreed or agreed 
toward the concept of an award for educators in hard-to-staff buildings. Principals and assistant 
principals/deans had the highest percentages with 74.6 percent and 73.8 percent, respectively 
(Appendix |, p. 40). 


What was the level of effectiveness for communicating information about the ASPIRE Award? 


For the May 2009 and subsequent survey administrations, there were ten items for which 
respondents rated the level of effectiveness regarding communication about the ASPIRE Award. 
Two of the ten items were added to the 2012 survey, and one item was added to the 2013 survey 
regarding effective communication. The responses are summarized in Table 10 (p. 30) using the item 
development as the baseline year. 


When comparing results from baseline to December 2014, nine of the ten areas of communication 
showed decreases. Knowing when specific information about my ASPIRE Award was available 
reflected the area of communication for which respondents indicated the highest increase for 
effectiveness (0.8 percentage point) (Table 10, p. 30). 


Based on the results of the May 2009 and 2014 surveys, 70.1 percent and 68.5 percent of 
respondents indicated that communication was moderately effective or very effective for knowing 
where to find information about my specific ASPIRE Award, reflecting the highest percentages for 
effectiveness for 2009 and 2014, respectively (Table 10, p. 30). 


Based on the December 2014 surveys, the areas for which the highest percentage of respondents 
perceived communications to be not effective or somewhat effective focused on providing clear 
explanations about comparative growth calculations (53.6 percent), providing clear explanations 
about value-added calculations (54.5 percent) and providing clear explanations about the award 
model (49.0 percent) (Table 10, p. 30). 


On the January 2014 survey, five questions were designed to determine how the respondents 
received specific types of communication. The results are summarized in Table 11 (p. 30). 


Based on the results of the December 2014 survey, 86.7 percent of respondents reported the ASPIRE 
e-mail as reflecting the highest percentage when compared to the other four methods used to 
communicate information about the ASPIRE Award program. This was followed by the ASPIRE 
eNEWS (69.9 percent) (Table 11, p. 30). 


When comparing whether respondents received/used any of the five different methods for 
communicating information about the ASPIRE Award program, 18.6 percent of respondents indicated 
Not Sure regarding Academic Services Memos, the highest percentage for this category. 
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What were the recommendations for changing the 2013-2014 ASPIRE Award suggested by 
respondents? 


Out of a total of 4,031 respondents on the December 2014 survey, 1,724 or 42.8 percent of the 
respondents provided at least one response for recommending changes to the 2013-2014 ASPIRE 
Award, whereas 57.2 percent of respondents did not provide any responses. Table 12 (p. 31) 
summarizes the frequency and percent of responses. 


A total of 3.1 percent and 4.4 percent of the 1,724 responses reflected that no changes were needed 
to the model or the response was simply, No Comment. The top six emergent categories reflected 
54.5 percent of the responses (Table 12, p. 31). 


The predominant suggestion centered on making the model equitable, fair, transparent, inclusive, 
with clear expectations so that all employees were treated equally, compensated equally, and/or had 
the opportunity to receive the same amount of award as the top dollar earners (12.8 percent). 
Elective/ancillary teachers, special education teachers, early childhood through grade 2, instructional 
support (i.e. counselors, librarians, and literacy coach), teaching assistants, and operational support 
staff (i.e. registrars, computer network specialists, and attendance specialists) were not eligible to 
receive the same level of compensation as core teachers with an EVAAS report. They felt “de-valued” 
by the way the model was designed. Some respondents indicated that the differences in eligibility 
and compensation were divisive for campuses. Moreover, respondents indicated that student 
success was a team effort, but the contribution of the team was not being equally valued for all 
members (Table 12, p. 31). 


Unintended consequences (divisive, cheating, free-riding) comprised about 12.0 percent of reponses. 
Respondents felt that some teachers would benefit from the award program but did not contribute 
significantly to student growth (free-riding). Another respondent indicated that “cheating is 
widespread in the HISD district...”, while another indicated that “The ASPIRE Award Program, as is, 
is not a fair way to compensate and reward those teachers who are effective in non-tested subjects.” 
Another respondent stated, “The very best teachers are leaving HISD for more competitive salaries 
and stipends.” Others have stated that it is “divisive”, and breeds frustration and confusion (Table 12, 
p. 31).” 


Approximantely 11.0 percent of the responses focused on the allocation of money. Respondents 
indicated that the money should be reallocated for student scholarships, smaller classes, better 
equipment, more tutors, school materials for students, clothes for students, attendance incentives for 
students, and to increase the base pay. Some respondents indicated that STAAR teachers or 
teachers in tested grade levels, teachers working in hard-to-staff schools and teachers providing 
instruction to low-income students and/or at-risk students should receive more money. Alternatively, 
respondents indicated that elective/ancillary teachers, special education teachers, Career and 
Technology teachers, librarians, nurses, early childhood teachers to grade 2 teachers (Group 2) 
should receive more money. Some respondents indicated that administrators should not receive any 
performance-pay money, their performance pay should be capped, or indicated that payouts for 
administrators were disproportionate in comparison to payouts for teachers. One respondent stated, 
“Making it more fair school wide for teachers. Central Office adminlistration] (including 
superintendent) should not be a part of ASPIRE” (Table 12, p. 31). 


A total of 174 responses or 6.9 percent of respondents were concerned about external factors that 
they perceived as impacting growth or the calcuation of growth. These were factors that teachers 
perceived as being out of their immedate control such as the classrooom composition (high numbers 
of behavior problems, English, Language Learners, at-risk, high performing, etc.). Another factor 
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centered on the time scheduled for a course. If students had more hours for reading and mathematics 
and less for science or art, how can they be compared? A third factor included those teachers who 
had tutors for their classes. Is it fair to calculate value-added scores for teachers with tutors and 
teachers who had no tutors. Student apathy and absenteeism were also factors that were out of a 
teacher’s control (Table 12, p.31). 


e Six percent of the respondents wanted to have the same earning opportunity as a core teacher with 
EVAAS®, or stated that their maximum award wasn’t commensurate with their professional 
contribution. 


e A total of 140 responses, or 5.5 percent of respondents to this question indicated that the ASPIRE 
Award be discontinued. One respondent cited, “Discontinue the program and use the ASPIRE money 
for scholarships for high achieving students to go to college." Another respondent stated, “Eliminate 
the program to restore true collaboration and teamwork” (Table 12, p. 31). 


Discussion 


The purpose of the 2013-2014 ASPIRE Award Survey was to gain insight regarding the level of knowledge 
and perceptions of Houston Independent School District (HISD) teachers and staff after eight years of 
implementation of growth-based performance pay in HISD, as well as their perceptions regarding the overall 
concept of performance pay. Additionally, participants had the opportunity to provide recommendations for 
making changes to the current model and for helping the district develop new teacher career pathways and 
a differentiated compensation system that rewards, recognizes, and retains effective teachers. This annual 
survey serves as a mechanism to gather valuable feedback from program participants. 


External factors, such as policy decisions, roll-out of a new model, or roll-out of any new model component 
may have influenced perceptions of growth-based performance pay since its inception. Although survey 
administrations followed the January payout with the exception of the 2014 survey administration when it was 
concurrent with the inquiry period, it is important to understand that eleven months had elapsed from the time 
of payout until the first survey administration (December 2007). Changes were instituted in the pay for 
performance model, communication about the model was enhanced, and training on the new model had 
commenced. Therefore, perceptions about the 2005-2006 Teacher Performance-Pay Model (TPPM) may 
have been influenced by anticipating these positive changes. 


On February 12, 2010 the Board of Education approved using value-added data as the 34th criterion to 
evaluate teacher effectiveness. Questions and uncertainties arose regarding the impact of this policy for 
teachers. When the 2008-2009 ASPIRE Award Survey was launched on February 23, 2010, amid this policy 
change, sufficient time had not elapsed to fully address questions or correct misconceptions. It is highly likely 
that the climate of concern that was evident among teachers during that time impacted their responses to the 
survey items. This is apparent in the decreases across the board in almost all items from 2009 to 2010. 


During the spring of 2011, budgetary shortfalls at the state level may have impacted perceptions and response 
rates during survey administration. Campuses were required to develop different budgetary plans, depending 
on the estimated shortfall in state funding that would result in the reduction in campus staff. Although final 
announcements were not made until April, an environment of speculation and uncertainty developed 
throughout all levels of the district. 


There were several factors that may have impacted the response rates for the 2012 survey. These included: 
multiple surveys targeting campus-based staff, including the ASPIRE Award and Career Pathways and 
Compensation Survey, administration of the new state assessment, State of Texas Assessments of Academic 
Readiness (STAAR), and rolling out the new teacher appraisal and development system. Teachers and other 
campus-based personnel were more focused and concerned about the new state assessment and the new 
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teacher appraisal system. The 2012—2013 school year marked the first year of inclusion of value-added and 
comparative growth measures formally introduced into the new teacher appraisal and development system. 


There have been four key areas that have shown mixed results over the past four to eight years. First, when 
comparing the survey response rate for December 2007 to the response rate for December 2014, there was 
an overall increase from 11.4 percent to 22.0 percent, but a decrease of 28.8 percentage points from May 
2009, 15.7 percentage points from March 2010, and 8.3 percentage points from March 2011. There was a 
decrease of 3.7 percentage points from January 2014. The response rate is low and caution is warranted in 
interpreting the data. 


Another key area, support for the program, showed mixed results over the eight-year period. Although the 
percentage of campus-based staff in favor or somewhat in favor of the concept of teacher performance pay 
decreased from 69.2 percent after the 2007 payout to 55.2 percent after the 2010 payout, this increased to 
58.6 after the 2012 payout, but then decreased to 49.7 percent with the December 2014 survey 
administration. When respondents were asked about their perceptions of the award model for that year, 44.4 
percent of respondents were in favor or somewhat in favor of the 2005-2006 Teacher Performance-Pay 
Model (December 2007) compared to the peak of 53.3 percent who were in favor or somewhat in favor of the 
ASPIRE Award program in May 2009. Alternatively, the majority of respondents have not been in favor or 
somewhat in favor of the ASPIRE Award program over the past five years. 


A related measure, support for the concept of differentiated pay, showed mixed results. Baseline data were 
collected during the May 2009 survey administration. Approximately 56.0 percent of respondents indicated 
they were in favor or somewhat in favor of differentiated pay in 2009, and this decreased to 48.3 percent in 
March 2010, but increased to 50.9 percent in March 2011, followed by an increase to 53.0 percent in March 
2012, but then decreased to 47.2 percent in March 2013, increased to 49.4 percent in January 2014, and 
was followed by a decrease to 48.1 percent in December 2014. 


The final key area centered on training sessions for value-added analysis. Historically, training courses have 
been offered on-line so that staff could complete the modules at their own pace. In addition, face-to-face 
training sessions were held around the district, and live webinars were offered to help teachers avoid travel 
and to be archived for future use. For the 2013-2014 school year, 31.1 percent of respondents indicated that 
they watched at least one of the Learning Modules on the SAS EVAAS® site in the last twelve months. 


Collecting feedback about effective communications was undertaken over the past six years to identify areas 
for improvement as well as areas that were effective. Based on survey results from 2009 to 2014, there was 
a decrease in effectiveness in nine of the ten areas for which data were available, including the newly added 
items, providing clear explanations about the award model, providing clear explanations about value-added 
calculations, and providing clear explanations about comparative growth calculations. Based on December 
2014 survey data, 53.6 percent of respondents indicated that communication was not effective or somewhat 
effective for providing clear explanations about comparative growth, 54.5 percent of respondents indicated 
that communications were not effective or somewhat effective for providing clear explanations about value- 
added calculations, and 49.0 percent of respondents indicated that providing clear explanations about the 
award model was not effective or somewhat effective. As value-added data and comparative growth data 
will now factor into all core teachers' appraisals, clear communication as well as effective training concerning 
them is a priority. 


When looking at the respondents by eligibility category, differences exist regarding how the ASPIRE Award 
program is perceived and the level of knowledge concerning the program. Administrators, such as principals 
and assistant principals/deans of instruction, indicate favorable perceptions concerning performance pay and 
their level of Knowledge, with the following exceptions: connection between classroom instruction and 
performance pay results, maximum ASPIRE award amount adequately recognized my efforts to increase 
student progress, and maximum ASPIRE award amount was commensurate with my _ professional 
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contribution. Core teachers have more positive perceptions than elective/ancillary teachers with one 
exception. Elective/ancillary teachers indicated that they had higher levels of knowledge regarding the 
ASPIRE Award program than core teachers (Groups 1-3). The differences in perceptions between core 
foundation teachers and non-core instructional staff have declined through time when looking at favorability 
in performance pay, student growth, and passing rates. 


For a performance pay system to be sustainable, the incentive amount has to be meaningful to all participants. 
Only 31.3 percent of principals agreed or strongly agreed that their maximum ASPIRE Award amount 
($15,000) was commensurate with their professional contribution. Teaching assistants indicated the highest 
percent for any category at 55.6 percent. Of the nine eligibility categories, instructional support staff and 
elective/ancillary teachers had the lowest level of agreement with regard to their maximum award amounts 
($1,350 and $3,000) at 14.3 percent and 17.5 percent, respectively. For those respondents that indicated 
they were not eligible to receive an award, only 24.8 percent agreed or strongly agreed that their maximum 
ASPIRE Award amount was commensurate with their professional contribution. On the 2014 survey, 
allocation of funding was the largest emergent category regarding recommended changes to the ASPIRE 
Award model, which included responses about increasing the award amount. 

The survey administered after each payout has served as a vehicle for respondents to recommend 
changes to the current model. Feedback is particularly valued to improve the ASPIRE Award program. Input 
varied from comments such as: “Too many changes to keep up with. Not enough people to explain how it 
actually works;” “We need a clearer understanding of how EVAAS® Scores are calculated. There are too 
many misconceptions about the process;” “Divide the ASPIRE Award amount equally among all teachers at 
a school. We all teach the same students. It should not be an award based on a subject area. It should be 
comprehensive for all on the same campus;” and, “I would not change it.” 
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Table 1. Nine Year Summary of Survey Response Rates by Pay for Performance Model 


Date of Survey # of Response 
Model and Year Administration Population Sample Respondents Rate 
2005-2006 TPPM December 2007 16,296 - 1,851 11.4 
2006-2007 ASPIRE Award May 2008 16,504 - 6,383 38.7 
2007-2008 ASPIRE Award May 2009 16,907 8,073 4,102 50.8 
2008-2009 ASPIRE Award March 2010 19,312 - 7,284 37.7 
2009-2010 ASPIRE Award March 2011 20,048 - 6,083 30.3 
2010-2011 ASPIRE Award March 2012 18,747 - 3,441 18.4 
2011-2012 ASPIRE Award March 2013 19,072 - 3,603 18.9 
2012-2013 ASPIRE Award January 2014 18,269 4,689 25.7 
2013-2014 ASPIRE Award December 2014 18,364 4,031 22.0 


Table 2. Number and Percent of Survey Respondents by Categorization, 2013-2014 ASPIRE Award, 


December 2014 Survey Administrations 


2012-2013 2013-2014 

Category N % N % 

Group 1, Core Teacher Grades 3-10 w/EVAAS 1,062 31.2 881 29.6 
Group 2, Core Teacher PK—2 702 20.6 535 18.0 
Group 3, Core Teacher Grades 3-12 w/o EVAAS 283 8.3 312 10.5 
Group 4, Elective/Ancillary Teacher 375 11.0 356 12.0 
Group 5, Instructional Support 253 7.4 259 8.7 
Group 6, Teaching Assistant 252 7.4 236 7.9 
Group 7, Operational Support 282 8.3 249 8.4 
Group 1L, Principals 104 3.1 74 2.5 
Group 2L, Assistant Principals/Deans of Instruction 90 2.6 70 2.4 
Total 3,403 100.0 2,972 100.0 


Highest Degree Held 


High School 133 3.4 
Some College 293 7.5 
Associate's Degree 141 3.6 
Bachelor’s Degree 1,420 36.4 
Some Graduate School 527 13.5 
Master’s Degree 1,272 32.6 
Doctoral Degree 111 2.8 
Total 3,897 100.0 
Average experience in HISD 10.3 
Average experience at current campus 6.8 
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Table 4. Number and Percent of Respondents Employed in HISD, Eligibility Status, Award 


Status, and Strand II Award Status 


Item Yes No N 
Were you employed in the Houston Independent 

School District during the 2013-2014 school 83.3 16.7 3,917 
year? 


Were you eligible to receive an ASPIRE Award 


for the 2013-2014 school year? Bee! ie ete 
Will you receive an ASPIRE Award for the 

2013-2014 school year (to be paid out in 58.9 41.1 2,889 
February 2015)? 

If you were a core teacher with an EVAAS® 

report, will you receive an individual 66.1 33.9 814 


performance ASPIRE Award? 


Table 5. Teaching in a Critical Shortage Area: Response Count and Response Percentage, 


December 2014 
Critical Shortage Area N % 
Special Education 384 11.8 
Bilingual Education 336 10.4 
English as a Second Language (ESL) 337 10.4 
Mathematics (Grades 6—12) 227 7.0 
Science (Grades 6-12) 176 5.4 
Spanish (Grades 6-12) 40 1:2 
N/A 1,744 53.8 
Total 3,244 100.0 


Table 6. Number and Percent of Survey Respondents Indicating Their Level of Agreement for which the 
ASPIRE Award Encouraged Specific Behaviors, May 2009 and December 2014 


Strongly Strongly 
Disagree! Neutral Agree/Agree 
Disagree 


N % % % 
The ASPIRE Award encourages meto:  Base- 2014 Base- 2014 Base- 2014 Base- 2014 
line* line* line* line* 


Continue teaching in the classroom 2,750 2,777 | 26.3 39.2 25.7 23.5 47.9 37.3 
Remain working in HISD 1,829 3,178 | 31.7 37.7 24.2 23.1 44.0 39.2 
Come to work on a daily basis 3,222 3,146 | 27.3 39.2 25.7 22.4 47.0 38.3 
Innovate in the classroom 2,846 39.2 22.8 37.9 
*Baseline year for the items /nnovate in the classroom was 2014 and Remain working in HISD was 2012; it was 2009 

for all other items. 
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Table 7. Number and Percent of Survey Respondents Indicating the Frequency of Selected 
Tak=id ql (oa d(o) ay-\ i od = (equ (ox:\-ymm DY=Yexs an] of-) @y-4eM A} 


Never Rarely Sometimes Often Always 

Over the past several years, | have 

N % % % % % 
Collaborated with my colleagues 2,484 0.1 0.7 3.7 20.8 74.7 
Used teacher-made assessments to 
make instructional decisions 2160 oF 16 nee aa Be 
Used data from district formative 
assessments (e.g. Snapshots or District 2,361 34 53 1A 255 54.3 
Level Assessments) to make instructional 
decisions 
Used standardized testing data to make 2,351 28 55 13.6 26.9 512 
instructional decisions 
Used value-added data to make 2,248 6.6 8.4 16.5 271 ALS 


instructional decisions 


Table 8. Number and Percent of Survey Respondents Indicating Their Level of Understanding for the 
ASPIRE Award Program and Its Components for the 2006-2007 and 2013-2014 ASPIRE 


Award, May 2008 and December 2014 Survey Administrations 


Please rate your level of Very 
understanding to the following Very Low/Low Sufficient High/High 
items: 


% % % 
2008 2014 2008 2014 2008 2014 
17.4 26.1 55.2 45.1 27.4 28.8 


5,844 3,137 | 21.3 33.5 50.0 40.9 28.7 25.6 


My understanding of ASPIRE is: 
My understanding of value-added 
analysis is: 

My understanding of the difference 
between student achievement and 5,848 3,145 | 11.6 17.8 43.9 43.4 445 38.7 
academic progress is: 

My understanding of how value-added 

information can help me as an 5,832 3,026 | 18.3 28.6 45.1 42.3 36.6 29.2 
educator is: 

My understanding of how to 
read/interpret value-added reports is: 
My understanding of the different 
components of the 2013-2014 5,835 3,117 | 23.2 33.8 48.7 42.5 28.1 23.6 
ASPIRE Award Program was: 

My understanding of how the ASPIRE 
Awards were calculated/determined is: 


5,817 3,073 | 23.7 29.8 47.0 42.6 29.3 27.6 


5,852 3,096 | 33.9 44.1 43.9 37.5 22.2 18.4 


HISD Research and Accountability 28 


Table 9. Number and Percent of Survey Respondents Indicating Their Perceptions About Award 
Amounts and the ASPIRE Award Model, March 2010 and December 2014 


Strongly Agree! 
Disagree! Neutral Strongly Agree 
Disagree 

N % % % 


2010 2014 2010 2014 2010 2014 # 2010 2014 
There is a connection between 
classroom instruction and ASPIRE 5,428 2,927 34.2 40.7 27.6 22.7 38.3 36.6 
Award results. 
The maximum award amount for my 
ASPIRE Award category adequately 
recognizes my efforts to increase 
student progress. 
The maximum award amount for my 
ASPIRE Award category encourages 
me to remain in a campus-based 
position. 
The maximum award amount for my 
ASPIRE Award category is 
commensurate with my professional 
contribution. 
The ASPIRE Award is a fair way of 
acknowledging a teacher’s impact on 5,417 2,977 46.6 46.1 26.6 21.7 26.7 32.2 
student growth. 
The formal inquiry process allowed me 
the opportunity to question the 4,812 2,519 22.8 23.3 39.7 31.6 37.5 45.1 
accuracy of my award. 
The ASPIRE Award should be 
continued with modifications 5,367 2,925 18.9 23.9 32.4 21.8 48.7 54.2 
incorporated on an annual basis. 


5,274 2,864 44.4 46.1 26.5 20.2 29.1 33.7 


5,319 2,899 37.2 42.6 32.4 24.4 30.3 33.0 


5,325 2,886 44.9 48.1 28.5 21.4 26.6 30.5 
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Table 10. Number and Percent of Survey Respondents Indicating Their Perceptions About Communicating 
Effectively, May 2009 and December 2014 


Not Effective! Moderately Effective/ 
N Somewhat Effective Very Effective 

Baseline 2014 Baseline 2014 Baseline 2014 
Knowing where to find information 
about the ASPIRE Award in general. ogee aoe Sze aos wee pee 
Knowing when specific information 
about my ASPIRE Award was 3,371 3,144 31.5 30.8 68.4 69.2 
available. 
Knowing where to find information 3,367 3,122 30.0 31.5 70.1 68.5 


about my specific ASPIRE Award. 
Knowing how to interpret and 
understand my specific ASPIRE 3,368 3,128 38.6 42.4 61.4 57.6 
Award Notice. 

Understanding the difference 
between submitting a question by e- 
mail versus submitting a formal 
inquiry about your final award. 
Understanding where to find 
information about the inquiry process 3,364 3,126 36.4 39.4 63.7 60.6 
on the portal. 

Understanding that formal inquiries 

were required to be submitted by a 3,352 3,129 34.7 35.1 65.4 64.9 
specific deadline. 

Providing clear explanations about 


3,362 3,124 38.6 42.1 61.4 57.9 


2,828 3,120 40.7 49.0 59.2 51.0 
the award model.* 
Providing clear explanations about 2,807 3,097 45.4 545 5A.7 45.5 
value-added calculations.* 
Providing clear explanations about 3,011 3,121 51.9 53.6 48.1 AGA 


comparative growth calculations** 
*Baseline year for the items asterisked was 2012, and **Baseline year was 2013; it was 2009 for all other items. 


Table 11. Number and Percent of Survey Respondents Indicating Their Receipt for Different 


Types of Communication December 2014 


Not Sure 
School Messenger (automated phone system) 3,097 


ASPIRE eNews 3,052 
Academic Services Memos (electronic format) 3,006 
ASPIRE e-mail 3,152 
ASPIRE portal 2,942 
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Table 12. Number and Percent of Responses for Recommended Changes and Educational 


Impact to the 2013-2014 ASPIRE Award, December 2014 


N % 
Make the model fair, transparent, equitable, inclusive, with clear expectations 323 12.8 
Unintended Consequences (divisive, cheating, free-riding, highly 301 119 
effective/etfective teachers leaving the district, negative culture) : 
Allocate more money for awards/allocate money for specified group(s)/reallocate 
money so that particular groups benefit and designated groups receive no award 
or their award is capped/allocate funds to buying resources, scholarships for 287 11.4 
students, smaller classes, more tutors, clothes for students, attendance 
incentives for students 
Factors perceived as impacting growth or the calculation of growth 174 6.9 
Same earning opportunity as highest award category/award not commensurate 
with professional contribution tat a0 
Discontinue the award 140 5.5 
Measuring growth/achievement (BOY/EOY/student growth/passing 
rates/campus, department, grade, subject, and/or individual award) ey as 
Change the Eligibility and Categorization Rules and make plant operators, 
janitors, food service, hourly employees, and tutors eligible/Attendance Rule 
(more days/eliminate)/Attendance bonus (reinstitute the bonus)/Don't include 
Appraisal Ratings (Biased in some cases) especially Student Performance 
Measures 
N/A or No Comment 110 4.4 
Training 105 4.2 
Performance measures or criteria (e.g. position in hard-to-staff school, number of 
highly effective teachers and retention of them, college readiness and college 99 3.9 
acceptance, parent's role, working with students new to the district) 
Don't Know/Not Sure 87 3.4 
Calculation/Formula 85 3.4 
No Changes/Satisfied 79 3.1 
Improve communications about the award/provide clearer explanations about the 
model and value added calculations/provide feedback for teachers based on their 69 27 
data/more timely communications about changes in the award model/teacher , 
input 
Pay Raise 62 2.5 
Appraisal 54 2:1 
Miscellaneous 45 1.8 
Payout Timeline/Value-Added Timeline 32 1.3 
Create a different model for non-core teachers/special education teachers 28 1.1 
Years of Experience & Advanced Degrees 16 0.6 
Linkage 10 0.4 
Inquiry Process 6 0.2 
Missing 2 0.1 
Total 2,524 100.0 
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APPENDIX A 
CROSS TABULATION SUMMARIZING THE PERCENT OF SURVEY RESPONDENTS INDICATING THE MAXIMUM FAVORABILITY TOWARD THE 
CONCEPT OF TEACHER PERFORMANCE PAY BY ELIGIBILITY CATEGORY, DECEMBER 2014 


100 


Row Percentages of Categories 
ol 
oO 


1 2 3 4 5 6 7 1L 2L Not Eligible 
Eligibility Category 
ln Favor/Somewhat In favor Neutral @ Opposed/Somewhat Opposed 


Key: 

Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 
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APPENDIX B 
CROSS TABULATION SUMMARIZING THE PERCENT OF RESPONDENTS INDICATING FAVORABILITY 
TOWARD THE CONCEPT OF TEACHER PERFORMANCE PAY BASED ON INDIVIDUAL STUDENT GROWTH BY ELIGIBILITY CATEGORY, DECEMBER 


2014 
100 
90 
wn 
2 80 
° 49.7 
® 70 
5 
w 60 
° 
& 50 
& 11.8 
- 40 
2 
a . 38.5 
5 20 
ax 
10 
0 
1 2 3 4 5 6 7 1L 2L Not 
Eligible 
Eligibility Category 
@In Favor/Somewhat In favor @ Neutral @ Opposed/Somewhat Opposed 
Key: 
Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 
Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 
Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 


HISD Research and Accountability 33 


APPENDIX C 


CROSS TABULATION SUMMARIZING THE PERCENT OF RESPONDENTS INDICATING FAVORABILITY 
TOWARD THE CONCEPT OF TEACHER PERFORMANCE PAY BASED ON PASSING RATES ONLY BY 
ELIGIBILITY CATEGORY, DECEMBER 2014 


100 


25.6 


Row Percentages of Categories 
ol 
oO 


1 2 3 4 5 6 7 1L 2L Not Eligible 
Eligibility Category 
In Favor/Somewhat In favor Neutral @ Opposed/Somewhat Opposed 


Key: 

Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 


HISD Research and Accountability 34 


APPENDIX D 


CROSS TABULATION SUMMARIZING THE PERCENT OF RESPONDENTS INDICATING FAVORABILITY TOWARD THE CONCEPT OF DIFFERENTIATED 
PAY BY ELIGIBILITY CATEGORY, DECEMBER 2014 


100 


Row Percentages of Categories 
aw 
oO 


1 2 3 4 5 6 7 1L 2L —_—Not Eligible 
Eligibility Category 


ln Favor/Somewhat In favor Neutral @ Opposed/Somewhat Opposed 


Key: 

Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 
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APPENDIX E 


CROSS TABULATION SUMMARIZING THE PERCENT OF SURVEY RESPONDENTS’ LEVEL OF 
UNDERSTANDING OF THE 2013-2014 ASPIRE AWARD PROGRAM, DECEMBER 2014 


100 


4.5 


Row Percentages of Categories 
ol 
oO 


40 
30 
20 
10 
0 
iT 2 3 4 5 6 7 1L 2L Not 
Eligible 


Eligibility Category 
BVery High/High Sufficient Very Low/Low 


Key: 

Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 
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APPENDIX F 
CROSS TABULATION SUMMARIZING THE PERCENT OF SURVEY RESPONDENTS’ INDICATING A CONNECTION BETWEEN CLASSROOM 


INSTRUCTION AND PERFORMANCE PAY RESULTS BY 
ELIGIBILITY CATEGORY FOR THE 2013-2014 ASPIRE AWARD PROGRAM, DECEMBER 2014 


100 


Row Percentages of Categories 
ol 
oO 


i 2 3 4 5 6 7 1L 2L Not 
Eligible 


Eligibility Category 


BAgree/Strongly Agree @ Neither @ Strongly Disagree/Disagree 


Key: 

Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 
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APPENDIX G 
CROSS TABULATION SUMMARIZING THE PERCENT OF SURVEY RESPONDENTS INDICATING THE MAXIMUM ASPIRE AWARD AMOUNT 


ADEQUATELY RECOGNIZED THEIR EFFORTS TO INCREASE STUDENT PROGRESS, DECEMBER 2014 
100 
90 
80 
70 
60 
50 
40 
30 


Row Percentages of Categories 


10 21.7 


1 2 3 4 5 6 7 iL 2L Not 
Eligible 


Eligibility Category 


@Agree/Strongly Agree @ Neither @ Strongly Disagree/Disagree 


Key: 

Group 1: Core Teacher Grades 3-10 wi/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 
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APPENDIX H 


CROSS TABULATION SUMMARIZING THE PERCENT OF SURVEY RESPONDENTS INDICATING THE MAXIMUM ASPIRE AWARD AMOUNT WAS 
COMMENSURATE WITH THEIR PROFESSIONAL CONTRIBUTION, DECEMBER 2014 


100 


VAR) 


10 : : : ; 22.3 


Row Percentages of Categories 


il 2 3 4 5 6 7 1L 2L Not 
Eligible 


Eligibility Category 
@Agree/Strongly Agree Neither Strongly Disagree/Disagree 


Key: 

Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 
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APPENDIX | 


CROSS TABULATION SUMMARIZING THE PERCENT OF SURVEY RESPONDENTS INDICATING FAVORABILITY TOWARD THE CONCEPT OF AN 
AWARD FOR EDUCATORS IN HARD-TO-STAFF BUILDINGS, DECEMBER 2014 


16.7 | ee fore | 16.2 ee ms Eee Ree Poe eee 
a ae | z 


19.1 in 31.7 aS 


1 


[o) 
jo) 


38 Bos Ef 19.7 9 195 


54.9 


Row Percentages of Categories 
ol 
oO 


a oe 
a ee ee ee ee | 
| ee ee ee ee 
ee ere ee fore ce ere 
| a ee ee en ee | 
| ee ee ee ee 
a ee ee ee | 


1 2 3 4 5 6 7 di: 2L Not 
Eligible 
Eligibility Category 


ln Favor/Somewhat In favor = Neutral = Opposed/Somewhat Opposed 


Key: 

Group 1: Core Teacher Grades 3-10 w/EVAAS® Group 6: Teaching Assistant 

Group 2: Core Teacher PK—2 Group 7: Operational Support Staff 
Group 3: Core Teacher Grades 3-12 w/o EVAAS® Group 1L: Principal 

Group 4: Elective/Ancillary Teacher Group 2L: Assistant Principal/Deans 
Group 5: Instructional Support Staff Not Eligible 


Note: Items that were skipped were coded as missing data and not included in the analysis. 


HISD Research and Accountability 40 


